Understanding American Agriculture: Challenges for the Agricultural Resource Management Survey by National Research Council of the National Academies

Author:National Research Council of the National Academies
Language: eng
Format: epub
Tags: Agriculture : Policy, Reviews and Evaluations
Publisher: NATIONAL ACADEMY PRESS
Published: 2007-11-07T00:00:00+00:00

Role of the Reinterview in Data Accuracy

The panel considers the control and measurement of data accuracy to be major issues for the ARMS data, especially for the cost-of-production and farm income figures. Some control and measurement methods are employed, and others, found useful in other settings and in prior incarnations of ARMS, are not.

As mentioned above, under the cooperative agreement, supervisors re-contact a sample of respondents in the process of conducting a case review. NASS reports that the quality control recontacts are randomly selected for each interviewer and supervisory interviewer. Additional recontacts are made if problems are suspected or uncovered. Quality control re-contacts are made by the field office survey statistician and by supervisory enumerators.

At the conclusion of the survey operation, as part of post-survey activities, the state statistical offices attempt to capture conceptual and reporting issues in the formal Survey Evaluations (Form E-2) that are forwarded to the national office of NASS. While important to ensure interviewer quality and provide insights on reporting problems, the current quality control procedure is no substitute for a program of systematic validation (reinterview) studies. There have been no reinterview studies conducted for ARMS in recent years.

A formal reinterview study is an important method for estimating and reducing nonsampling errors in surveys (Biemer and Forsman, 1992). It can, like the current recontact program in ARMS, evaluate the fieldwork by detecting and discouraging cheating and interviewer errors. It can also play a more comprehensive role in ferreting out nonsampling error by identifying content errors that should be reflected in a model of survey errors by estimating (a) simple response variance, or the variability in the survey estimate over conceptual repetitions of the survey, and (b) response bias, or the response errors that would be consistent over repetitions of the survey. Content errors include definitional problems, misinterpretation of questions and survey concepts, and reporting errors.

NASS has a rich history as a pioneer in implementing recurring reinterview studies in the agricultural surveys (Hanuschak et al., 1991). As early as 1975, formal reinterview studies were conducted to probe how well reporting unit concepts were understood by the respondent. (Probing questions in this early study found that approximately 30 percent of respondents incorrectly reported total acres operated, and 20 to 30 percent incorrectly reported specific livestock inventories [Bosecker and Kelly, 1975].) From the late 1980s to the early 1990s, several important reinterview studies were conducted to measure response bias in surveys that included quarterly grain stocks, the June agricultural survey, on-farm grain stocks, hogs, and cattle on feed. These surveys not only identified conceptual difficulties that could be remedied by changes in questions or in interviewer instructions and training, but also had the more practical application of informing the board estimation process with measures of response bias. Although expensive in terms of statistician and interviewer time and additional burden on the part of respondents, these formal reinterviews were considered a success and formed the basis for a panoply of recommendations for a more robust reinterview program in the future.

Particularly

Download

Copyright Disclaimer:
This site does not store any files on its server. We only index and link to content provided by other sites. Please contact the content providers to delete copyright contents if any and email us, we'll remove relevant links or contents immediately.